Best practices for private online panels

By Jerry Thomas | March 1, 2021

Reading time: 9 minutes

Abstract

From recruiting it to maintaining it, Jerry Thomas gives guidance on how to set up a private online panel.

Research Topics:: Consumer Research | Panels-Online | Qualitative Research | Quantitative Research
Industry/Market Focus:: Consumers
Content Type: Magazine Article

Share Print

Getting the conversation started

Editor's note: Jerry Thomas is president and CEO of Decision Analyst, an Arlington, Texas, research firm.

Over the past decade, many corporations have set up private online panels or online communities as an economical way to conduct surveys and qualitative projects. A private research panel (sometimes called a custom panel, customer panel or proprietary panel) is created by a company solely for its own use. The company recruits and maintains the online panel and uses the panel exclusively for its own research purposes. While many private online panels are a success, many are deemed failures. The purpose of this article is to outline some guidelines and best practices for private online panels.

Private panel versus online community

A private online panel is sometimes confused with an online community. Typically, a private online panel is made up of thousands of participants, while an online community tends to be much smaller (i.e., hundreds of members). A private panel tends to have a long life; it can be successfully operated for years. A community, however, is often limited in life (maybe a few months or a year) to minimize the risks of respondent conditioning. Private panel members are rarely permitted to interact with other panel members but an online community might allow its members to interact.

Online communities tend to be less expensive to operate than private panels because the number of participants is smaller and the same community members are used over and over again. Sampling controls are minimal for communities but they can be quite rigorous for private panels. As a best practice, it is probably wise to think of communities as qualitative research and to think of private panels as quantitative research (although both can reverse roles at times).

Budget pressures

The growth in the number of private panels is largely a function of a slow-growth world economy and the corresponding budget pressures within large corporations. It is the lure of cost-efficiencies that prompts most companies to consider private panels. Under optimal conditions, a private online panel can reduce the cost of a typical research project by 25 to 35 percent.

Falling response rates to online commercial access panels is also a major reason for the adoption of private panels (where response rates tend to be higher). Another major factor is category incidence. If a company’s product is used by only 1 percent of the population or less, then a private panel of customers might be the only affordable way to do research. Another consideration is the degree of consumer interest in the product category. High-interest product categories are generally better candidates for online private panels but even low-interest categories can often benefit from private panels.

Limited value

Private online panels are not appropriate for all types of research. If marketing directors are trying to survey a representative sample of all adults, then the private panel is of limited value since it most likely contains only customers or those interested in the brand (subsets of the total adult population). Many studies, however, can be conducted among customers or those interested in the company’s brand; it’s not a perfect sample but often it is good enough to help make the correct decision. It is also possible to recruit non-customers into private panels but the recruiting costs for non-customers can be quite high.

One of the first questions marketing directors must ask is, “Will my company really use the private panel once it is set up?” We have set up a number of private panels for clients with big research plans that never materialized. Each one planned to use their private panel frequently but the burdens and mechanics of proposing studies to internal audiences, getting approvals, preparing purchase orders, writing questionnaires, pulling samples, tabulating the results and trying to write reports bogged the whole process down. Therefore, few studies were actually conducted and these private panels were never economically viable.

The economic feasibility of private panels all depends on the financial details. Every product category is different, so marketing directors will have to sit down and do the math for their particular situation. As a rule, if a company does not conduct at least one research project per month, a private panel will rarely make economic sense. That is, the expense of setting up and maintaining the private panel will be greater than the savings on the research studies.

Another consideration is the subject matter of the research. High-interest product categories are the best candidates for private panels. A panel of golfers or pilots or sailboat captains or motorcycle owners will work like a charm because the respective panel members are very interested in the categories. It would be easy to recruit panel members and participation rates in surveys would be high.

Conversely, if marketers want to set up a panel of insurance owners or water-utility customers or banking customers (comparatively low-interest categories), it will be much more difficult to recruit panel members and response rates to surveys will be low.

Lastly, the lower the incidence of a product category, the greater the potential savings from a private panel is likely to be. For example, if a product category happened to be experimental aircraft, the percent of the population (i.e., incidence) of those who build experimental aircraft is far less than 1 percent of the U.S. adult population. If a company had to screen large probability samples of U.S. adults to find experimental-aircraft builders, every study would cost a small fortune. But if marketers owned a private panel of these aircraft enthusiasts, studies could be conducted at modest costs.

Advantages

If a private online panel is set up and managed properly, its advantages are:

Better decisions. More marketing questions can be addressed and answered, based on objective consumer feedback.

Economy. Surveys via private online panels typically save 25 to 35 percent versus the price of traditional online surveys. The savings are even greater for low-incidence categories.

Speed. Most surveys are turned around in 14 days (from questionnaire design through final written report). Complicated projects take longer, of course. Having the private panel in place – and all the paperwork and approvals taken care of – means that a company can launch a survey almost instantly in the event of an emergency or urgent need.

Accuracy. Private online panels can provide reasonable, balanced samples (demographically and geographically).

Response rates. Private panels tend to enjoy much higher response rates, compared to commercial online access panels. This could be a greater advantage in the future if response rates for commercial panels continue to fall.

Longevity. Private panels can last for years if new panelists are added on a regular basis and older panelists are removed from the panel.

Best practices

Companies tend to reap the greatest benefits from a private online panel when it is properly managed and its use is encouraged. To achieve best practices, marketing directors should:

Assign one person to be responsible for scheduling, expediting and coordinating with internal groups or brand teams so as to keep research projects moving ahead.
Consider quarterly omnibus surveys where each group or brand can submit a question or two.
Charge the private panel’s cost to a central marketing budget so that it is “free” to individual brands or groups. If the studies are free, brands are eager to do as many studies as possible (and thereby make better decisions).
Simplify the approval process so that a group or brand team can launch a private panel survey with minimum paperwork (no purchase orders, no elaborate approval processes).
Standardize often-repeated types of studies so that the organization learns over time what the answers mean.
Create a database of normative data for the standardized studies so that results have greater value and meaning.

Internal staff versus outsourcing

If a marketing director decides to set up a private panel, should she do it herself with internal staff or outsource it to a research firm with experience in private panels? The larger the company and the greater the number of projects, the more likely it is marketers should staff up and do it themselves. If they plan to do more than 100 or 150 research projects a year, then building a professional internal staff might make sense. Remember, marketing directors will need to hire some individuals with research and supplier operations experience – folks who really understand the nuts and bolts of panel management, sampling, questionnaire design, coding, tabulation, quality assurance, data security and e-mail deliverability.

Also, the company will need an array of software to support the operation of a private panel. If a company is likely to use a private panel for fewer than 100 projects a year, it should probably outsource the private panel to a research company with expertise in private-panel development and management. The research firm will manage the panel, answer e-mails and phone calls, help keep projects on schedule, provide all the software and programming, design questionnaires, conduct the surveys and write reports for the company.

Purchasing a turnkey private panel service frees up marketing directors to be the research consultant within the organization, rather than the research mechanic. Directors can operate with minimal staff, since most of the work is performed by the private-panel partner.

Recruiting tips

Recruiting a private panel can be a challenge. The best way is to collect customer e-mail addresses is at every point of contact with customers. Then the company will have a massive database of customer e-mail addresses from which to recruit for a private panel. If the company does not have customer e-mail addresses, then panel recruiting tends to be expensive. Marketing directors can insert panel invitations in packages. They can mail letters to targeted groups inviting them to join. They can purchase opt-in e-mail lists. They can post a link on their website. They can send out publicity releases announcing their panel. The best methods of recruitment vary by product category and marketing directors will have to experiment to identify the most cost-effective methods.

Threats and risks

Once a marketing staff starts recruiting for a private panel, they will need to be careful that a few panel members don’t join the panel multiple times or complete the same survey more than once. A best practice is to use digital fingerprinting to identify duplicate respondents. A major risk is the potential liability resulting from hackers gaining access to panelists’ personally identifiable information. Another best practice is to encrypt all panelists’ data at a high level of security. If large numbers of e-mails are sent to panelists via a corporate e-mail system, marketers might find some internet service providers (ISPs) shutting down delivery of the corporate e-mail. A best practice is to set up a completely separate system to handle private panel e-mails. Even so, there is always a risk that some ISPs will block delivery of e-mails to panelists, so it is important to track e-mail delivery and response rates by ISP.

Another risk is panel attrition. If the panel is not nurtured and used, panelists will quickly lose interest and drop out. Best practices here include sending birthday cards and other communications to panelists, answering all e-mails and phone calls promptly and using each panelist 10 or more times per year. Respondent fatigue is also a risk if questionnaires are long and tedious or poorly designed. A best practice is to keep questionnaires short and concise. Longer questionnaires are possible for some categories and incentives can make longer questionnaires acceptable to participants.

One final risk in some countries revolves around accessibility. If your corporate communications promise equal access to all, regardless of their disabilities or handicaps, then your surveys must be accessible (a very high standard to meet). Legal risks can be significant in these instances.

Just one tool

A private online panel can be a useful tool in the research toolbox but it is just one tool. In-house marketing directors will still need professional research agencies to help the company with studies that require probability samples (market segmentation, benchmarking, volumetric forecasting, etc.). They will need research suppliers to help with central-location studies, in-person interviewing and qualitative research. They probably will still need professional research firms to help with very complex studies (choice modeling, segmentation, optimization, etc.). They will also need professional research companies to help with major strategic projects where objectivity and outside perspective are crucial. However, for the day-to-day tactical studies among a company’s customers, private online panels may be an option to consider.